Skip to content

feat(viser): overlay sim ground truth - #3835

Draft
mustafab0 wants to merge 2 commits into
mainfrom
feat/grasp-demo-viser-ground-truth
Draft

feat(viser): overlay sim ground truth#3835
mustafab0 wants to merge 2 commits into
mainfrom
feat/grasp-demo-viser-ground-truth

Conversation

@mustafab0

@mustafab0 mustafab0 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • publish configured MuJoCo body poses on a sim-only typed stream
  • extend VisualizationSpec with a truth/belief overlay hook
  • render faint cyan truth ghosts beside solid planning obstacles in Viser
  • show a toggle, legend, and live truth-vs-belief delta table
  • enable Viser and six-object truth for xarm-room-sim

@greptile-apps

greptile-apps Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This change adds an optional simulator ground-truth overlay to the Viser manipulation view. It publishes configured MuJoCo body poses, forwards them to the visualization, renders cyan truth markers with visibility controls and a truth-versus-belief distance table, and enables the feature for the xArm room simulation.

No defect was reproduced.

T-Rex validation blocked

  • Missing package: The targeted Python harness could not import the project runtime because the git Python module from the GitPython package is unavailable.
  • Missing tool: The xArm room simulation asset is stored as a Git-LFS pointer and git-lfs is unavailable, so the configured MuJoCo body names could not be checked against the real room scene.

Confidence Score: 5/5

No verified implementation defect blocks merging.

There are no final severity-bearing findings. Source tracing confirms that matching typed pose streams, visualization forwarding, overlay configuration, marker lifecycle, and cleanup handling are present; the environment prevented execution against the packaged xArm room scene.

Files Needing Attention: If the simulation environment is available, dimos/simulation/engines/mujoco_sim_module.py and dimos/robot/manipulators/xarm/blueprints/simulation.py merit a final runtime check against the Git-LFS-backed xArm room asset.

T-Rex T-Rex Logs

What T-Rex did

  • I ran a fake-backed verification harness to exercise the generated stream connection, MuJoCo pose publication and rate limiting, Viser marker lifecycle, and report forwarding to the visualizer, but import resolution stopped at a missing git Python package before assertions could run.
  • I attempted to inspect the xArm room asset, but it is a Git-LFS pointer and the git-lfs tool is unavailable.
  • The harness run produced no severity-bearing findings and was blocked before assertions due to tooling limitations.

T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "feat(viser): overlay sim ground truth" | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant